Trend Test_v1

Mann-Kendall Trend Test Formula

The Mann-Kendall test (using Kendall's Tau) detects monotonic trends in time series data.

Kendall's Tau (τ) Formula:

$$\tau = \frac{C - D}{\frac{1}{2}n(n-1)}$$

Where:

• $C$ = Number of concordant pairs
• $D$ = Number of discordant pairs
• $n$ = Number of data points

Concordant vs Discordant Pairs:

For all pairs of observations $(x_i, y_i)$ and $(x_j, y_j)$ where $i < j$:

• Concordant: if $(x_j - x_i)$ and $(y_j - y_i)$ have the same sign
• Discordant: if $(x_j - x_i)$ and $(y_j - y_i)$ have opposite signs

$$C = \sum_{i<j} \text{sign}(t_j - t_i) \cdot \text{sign}(Y_j - Y_i) = 1$$ $$D = \sum_{i<j} \text{sign}(t_j - t_i) \cdot \text{sign}(Y_j - Y_i) = -1$$

Test Statistic (Z-score):

For large samples ($n > 10$), the significance is tested using:

$$Z = \frac{\tau}{\sigma_\tau}$$

Where the standard error is: $$\sigma_\tau = \sqrt{\frac{2(2n+5)}{9n(n-1)}}$$

P-value Calculation:

The p-value is derived from the Z-statistic using the standard normal distribution: $$p\text{-value} = 2 \times P(Z \geq |z|)$$

Hypothesis Testing:

• $H_0$: No monotonic trend exists (τ = 0)
• $H_1$: A monotonic trend exists (τ ≠ 0)

Decision Rule (α = 0.05):

• If $p\text{-value} < 0.05$:

  • Reject $H_0$ → Significant trend exists
  • If $\tau > 0$: Increasing trend
  • If $\tau < 0$: Decreasing trend
  • Station PASSES (trend detected)

• If $p\text{-value} \geq 0.05$:

  • Accept $H_0$ → No significant trend
  • Station FAILS (no trend detected)

Kendall's Tau Interpretation:

• $\tau = +1$: Perfect positive correlation (always increasing)
• $\tau = 0$: No correlation (no trend)
• $\tau = -1$: Perfect negative correlation (always decreasing)
• Typical range: $-1 \leq \tau \leq +1$

Step-by-Step Analysis Process

Step 1: Data Loading

• Load precipitation data from Excel file
• Data contains a 'Date' column and multiple grid/station columns with precipitation values

Step 2: Time Series Preparation

• Create sequential time index (0, 1, 2, ..., n-1) representing time progression
• Each row represents a time point in chronological order

Step 3: Mann-Kendall Trend Test

For each precipitation station/grid column:

1. Calculate Kendall's Tau (τ) with scipy.stats import kendalltau

   • Correlate time index with precipitation values
   • Measures monotonic relationship between time and precipitation

2. Calculate P-value

   • Determines statistical significance of the trend
   • Tests if observed τ could occur by random chance

3. Interpret Results

   • If p-value < 0.05 (significant trend):

     • τ > 0 → Increasing precipitation trend
     • τ < 0 → Decreasing precipitation trend
     • Station PASSES eligibility test

   • If p-value ≥ 0.05 (not significant):

     • No monotonic trend detected
     • Station FAILS eligibility test

Step 4: Summary Generation

• Compile results for all stations into summary DataFrame
• Include: Station name, Tau value, P-value, Trend direction, Pass/Fail status

Step 5: Export Results

• Save summary to Excel file for further analysis
• Display results in notebook for immediate review

Expected Output

A table showing which stations exhibit significant precipitation trends over time, helping identify eligible stations for further analysis.

import pandas as pd
import numpy as np
from scipy.stats import kendalltau

# Load your Excel file (assuming it's named 'data.xlsx')
file_path = r"\Data\grids_precipitation_data.xlsx"
df = pd.read_excel(file_path)

# Initialize an empty list to store the summary data
summary_data = []

# Reset summary data list
summary_data = []

# Create time index (sequential values from 0 to n-1)
time_index = np.arange(len(df))

# Iterate through each column (skip 'Date' column)
for col in df.columns:
    if col == 'Date':
        continue
    
    # Perform the Mann-Kendall test (correlation between time and values)
    tau, p_value = kendalltau(time_index, df[col])
    
    # Check if the trend is significant (p-value < 0.05)
    if p_value < 0.05:
        if tau > 0:
            trend = 'Increasing'
        else:
            trend = 'Decreasing'
        trend_passed = 'Yes'
    else:
        trend = 'No significant trend'
        trend_passed = 'No'
    
    # Append the result to the summary data list
    summary_data.append({
        'Column': col, 
        'Tau': round(tau, 4),
        'P-value': round(p_value, 4),
        'Trend': trend,
        'Significant (p<0.05)': trend_passed
    })

# Create a new DataFrame from the summary data
summary_df = pd.DataFrame(summary_data)

# Save the summary to a new Excel file
summary_file_path = r"\trend_test_results.xlsx"
summary_df.to_excel(summary_file_path, index=False)

print(f"Summary saved to {summary_file_path}")
print(summary_df)